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OBTAINING  FEEDBACK  ON  THE  LIFE  CYCLE  COST  ANALYSIS  OF  THE 
STANDARD  ENGINE  TEST  SYSTEM 

The  Naval  Air  Warfare  Center  Aircraft  Division  is  the  cognizant  field  activity  for 
the  Navy's  engine  test  systems.  The  Cost  Analysis  and  Estimating  Section 
completed  a  life  cycle  cost  analysis  for  the  Standard  Engine  Test  Systems 
(SETS)  which  is  the  Navy's  latest  generation  engine  test  system.  A  conservative 
LCC  savings  estimate  of  $862M  was  generated  even  though  the  acquisition  cost 
for  the  sets  is  much  higher  than  the  acquisition  cost  for  the  engine  test  systems 
that  it  replaces. 

Determining  the  accuracy  of  a  LCC  analysis  is  important.  Generation  of  cost 
element  relationships  (CER),  quality  data  bases,  and  proper  calibration  of  costs 
estimating  processes  is  important  to  ensure  the  validity  of  future  estimates  and 
maintaining  the  confidence  of  the  program  managers. 

This  paper  will  focus  on  the  verification  of  the  results  of  the  initial  report  and 
offer  a  comparison  between  the  expected  and  actual  life  cycle  costs.  The  lead 
time  for  support  equipment  is  shorter  than  many  of  the  larger  more  complex 
systems  and  products,  thereby  allowing  us  the  ability  to  develop  cost 
performance  measurement  results.  The  presentation  will  conclude  with  a 
"lessons  learned"  discussion  for  LCC  analyses. 

John  Melin 
SR44 

Naval  Air  Warfare  Center 
Aircraft  Division 
Lakehurst,  NJ  08733 
(908)  323-1494 
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OF  THE  STANDARD  ENGINE  TEST  SYSTEM 


The  opinions  or  assertations  contained  herein  are  the  private 
ones  of  the  writers,  and  are  not  to  be  construed  as  official  or 
reflecting  the  views  of  the  Navy  Department  or  the  Naval  Services 
at  large. 


4 


TABLE  07  CONTENT8 


1  INTRODUCTION  1 

2  SETS  HISTORY  1 

3  TEST  STAND  HARDWARE  3 

4  TEST  STANDS  4 

5  SETS  LIFE  CYCLE  COST  5 

6  STATUS  QUO  COST  7 

7  SETS  DECISION  10 

8  SETS  TEST  PLAN  11 

9  INITIAL  FEEDBACK  OF  SETS  LCC  11 

10  ANALYSIS  &  COMPARISON  OF  EXPECTED/ACTUAL  LCC  15 

11  LESSONS  LEARNED  15 

12  CONCLUSION  16 

APPENDIX  A  METHODOLOGY  OF  ESTIMATE  17 

APPENDIX  B  ACRONYMS  18 

APPENDIX  C  LIFE  CYCLE  DEFINITIONS  19 

APPENDIX  D  SETS  ASSUMPTIONS  22 

APPENDIX  E  ETS  ASSUMPTIONS  24 

SOURCES  CONSULTED  28 


A 


♦ 


LZ8T  OF  FIGURES 

1  STANDARD  ENGINE  TEST  SYSTEM  OPERATIONAL  LAYOUT  2 

2  ENGINE  TEST  SYSTEMS  5 

3  ENGINES  TO  BE  TESTED  ON  SETS  6 

4  SETS  LIFE  CYCLE  COSTS  7 

5  ETS  LIFE  CYCLE  COSTS  8 

6  LAND  BASED  ETS  INVENTORY  8 

7  CARRIER  BASED  ETS  INVENTORY  9 

8  STATUS  QUO  VS.  SETS  11 

9  SETS  PREDICTED  ACQUISITION  COSTS  13 

10  SETS  PREDICTED  SPARE  COSTS  13 

11  SETS  PREDICTED  MANPOWER  COSTS  14 

12  SETS  PREDICTED  FUEL  COSTS  14 


4 


OBTAINING  FEEDBACK  ON  THE  LIFE  CYCLE  C08T  ANALYSIS 
OF  THE  STANDARD  ENGINE  TEST  8Y8TEK 

1.  INTRODUCTION 

Life  Cycle  Cost  (LCC)  estimating  have  become  an  increasingly 
important  part  of  the  DoD  acquisition  approval  process.  This  is 
especially  true  under  the  DoD  5000  Series  instructions.  In 
addition  to  traditional  techniques  of  LCC  estimating ,  the  use  of 
Parametric  Modeling  has  increased  due  to  the  availability  of  the 
Personal  Computer  (PC) .  While  much  research  has  been  done  to 
develop  Parametrics,  the  authors  maintain  that  there  is  a  need 
for  the  cost  analysis  community  to  review  the  actual  results  of 
each  Life  Cycle  Cost  (LCC)  in  order  to  improve  each  new  estimate. 
With  ACAT  I  through  IV  programs,  obtaining  feedback  on  the 
initial  LCC  estimate  takes  years.  This  is  due  to  the  long 
approval  cycle,  development,  production,  and  logistics  lead 
times.  It  can  also  be  noted  that  the  complexity  and  cost  of  the 
typical  DoD  program  is  increasing.  At  this  point  in  time,  there 
is  a  lack  of  programs  which  have  had  LCC  estimates  performed 
using  parametric  modeling  on  the  PC  which  are  ready  to  have  the 
results  examined. 

2 .  SETS  HISTORY 

The  Naval  Air  Warfare  Center  Aircraft  Division  Lakehurst 
(NAWCADLKE)  is  the  Cognizant  Field  Activity  (CFA)  for  Navy  Engine 
Test  Systems  (ETS) .  Navy  aircraft  engine  maintenance  is 
performed  under  the  OPNAVINST  4790.2  series  and  is  performed  at 
Organization,  Intermediate,  and  Depot  levels.  A  significant 
amount  of  money  has  been  spent  on  engine  test  systems  at  I  level 
since  the  gas  turbine  engine  replaced  reciprocating  engines  in 
Navy  aircraft. 

By  the  late  1980's,  it  was  becoming  apparent  that  a  large 
number  of  the  test  systems  were  becoming  obsolete  and  would 
require  replacing  by  the  year  2000.  As  seen  in  figure  1,  many  of 
the  components  in  a  typical  test  system  are  functionally  the 
same.  A  standard  replacement  for  all  of  the  test  systems  needed 
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study.  In  1990,  the  first  Life  Cycle  Cost  (LCC)  analysis  for  the 
Standard  Engine  Test  System  (SETS)  was  performed  and  showed  a 
significant  cost  savings  to  the  Navy  if  a  standardized  engine 
test  system  were  developed.  The  main  life  cycle  cost  savers  were 
manpower  and  fuel  although  other  areas  also  showed  savings. 

To  provide  state-of-the-art  capability  for  testing  current 
and  future  turbine  engines,  a  Standard  Engine  Test  System  (SETS) 
was  proposed  to  replace  existing  U.S.  Navy  aircraft  ETS.  The 
1990  report,  which  was  revised  in  1991,  provided  a  life  cycle 
cost  analysis  of  the  proposed  SETS  versus  the  current  test  cell 
equipment.  A  value  analysis  of  the  existing  ETS  was  also 
presented.  The  report  also  included  a  cost  estimate  for 
developing  one  (1)  SETS  prototype  and  producing  of  132  SETS 
units.  It  addressed  the  method  of  cost  estimating  employed, 
defines  the  cost  categories  examined,  provided  a  comprehensive 
analyses,  and  confirmed  reported  conclusions.  A  1991  report  gave 
a  fuel  consumption  estimate  for  the  SETS. 

3.  TEST  STAND  HARDWARE 

The  SETS  is  the  first  of  a  new  generation  of  I-level  engine 
test  equipment  capable  of  testing  all  types  of  gas  turbine 
engines  including  turbojet,  turbofan,  turboshaft,  turboprop, 

APUs,  GTCs,  etc.  It  consists  of  a  programmable  engine  control 
system,  as  well  as  interface  adapters,  cable  assemblies,  and 
engine  adapters  and  stands.  The  SETS  has  the  capability  of 
operating  in  a  closed-loop  (automatic)  mode,  open-loop  (manual) 
mode,  or  in  a  "follow-me”  mode  depending  upon  software 
capability,  operator  preference,  or  local  test  procedure/ 
requirements.  The  operator  control  panels  on  the  SETS  consists 
of  toggles,  push-button  switches,  and  a  touch  screen  "soft*1 
engine  control  module  (SECM) .  As  the  engine  is  operated 
throughout  its  performance  envelope,  data  points  are  monitored 
and  stored  in  the  computer  memory  to  be  printed  out  and  plotted 
at  the  completion  of  the  test  run.  Two  19-inch  high-resolution 
color  monitors  provide  the  operators  with  precise  digital  and 
graphic  displays  of  all  parameters  which  require  monitoring. 
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The  SETS  is  an  outgrowth  of  Engineering  Specification  1290, 
which  defined  the  requirements  for  an  Integrated  Computer 
Automated  Test  System  (ICATS)  for  the  Joint  Services  Cruise 
Missile  Program.  The  heart  of  the  ICATS  is  the  Automatic  Data 
Acquisition  and  Processing  System  (ADAPS) ,  an  engine  test  control 
system  already  in  wide  commercial  use  for  automated  testing  of 
gas-turbine  engines.  The  main  design  effort  of  the  SETS  is  to 
revise  and  repackage  the  ADAPS  into  the  SETS  configuration. 
Modification  of  the  ADAPS  include: 

a.  Engine  Control  Module  (ECM)  -  A  soft  (programmable)  engine 
control  module  (SECM)  must  be  designed  and  integrated  into  the 
ADAPS  system  in  order  to  test  different  engine  types. 

b.  Facilities  Control  Module  (FCM)  -  The  FCM  must  be  modified  so 
that  it  is  capable  of  controlling  all  non  engine  sub-assemblies 
and  support  equipment  used  in  conjunction  with  the  engine  during 
engine  test. 

c.  Junction  Box  -  The  junction  box/engine  interface  panel  must  be 
changed  to  the  common  Navy  design  as  delineated  by  drawing 
604AS401  for  the  purpose  of  using  existing  Navy  engine  umbilical 
cable  assemblies. 

d.  Shelter  -  A  portable  shelter  to  house  the  SETS's 
Instrumentation  and  Control  Console  Assembly  and  other  facilities 
related  subsystems. 

4.  TEST  STANDS 

At  the  present  time,  there  are  a  total  of  twenty  two  (22) 
different  engine  test  systems  in  the  Navy  inventory.  These 
systems  are  given  as  figure  2  below  and  exclude  Navy  depot  engine 
test  systems,  which  are  beyond  the  scope  of  this  report. 
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ENGINE  TEST  8Y8TEMS 


|  ENGINE  TYPE 

TEST  8YSTEM 

TURBOFAN/JET  ET8 

A/W37T-1 

A/F32T-6 

A/F32T-6A 

A/F32T-9 

A/F32T-10 

A/F32T-11 

A/E37T-14 

A/M37T-23 

TURBOSHAFT  ET8 

A/W37T-2 

A/F37T-16 

(V1-V4 ) 

A/M37T-17 

A/M37T-18 

(V) 

A/E37T-24 

21C1250 

21C2027 

21C2222 

(V2-V6) 

TURBOPROP  ETS 

A/E37T-17 

(V) 

A/F37T-19 

(VI) 

SMALL  GAS  TURBINE  ETS 

A/E37T-20 

A/E37T-20A 

A/E37T-26 

A/E37T-26A 

FIGURE  2 


Figure  3  below  gives  the  engines  that  were  proposed  to  be 
tested  on  the  SETS  at  the  time  that  the  original  LCC  report  was 
written.  Due  to  uncertainty  at  the  time  the  report  was  written, 
engines  such  as  the  J57,  J60,  and  J79  were  included  in  the 
report.  The  F-405  engine  was  excluded  because  the  T-45  aircraft 
program  is  using  contractor  support.  It  is  possible,  but  still 
not  decided  if  the  SETS  will  be  used  to  support  the  F-405  engine 
at  a  later  date. 


ENGINES  TO  BE  TESTED  ON  SET8 


|  ENGINE  TYPE 

ENGINE 

TURBOJET 

J52 

J57 

J60 

J69 

J79 

J85 

DART  MK5298X 

M861 

YJ400-WR-400 

TURBOFAN 

TF30 

TF34 

TF4 1 

F110 

F402 

F412 

JT8D-9 

F404 

CFM56 

TURBOPROP 

T56 

PT6A 

TURBOSHAFT  T53 


T58 

T64 

T406 

T700 


GTCP36-200/201 

T-62T-11/27 

T-62T-40-1 

T-62T-46-1 

T-62T-47-1 

GTC95-2 

LUCAS  MK  II/IV 


Figure  3 


APU  &  GTC 


5.  SETS  LIFE  CYCLE  COST 

The  methodology  for  determining  the  SETS  LCC  is  provided  as 
appendix  A  of  this  report.  A  combination  of  analogy  and 
parametric  based  techniques  were  used.  Results  of  the  estimate 
are  given  as  figure  4  below. 
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SETS  LIFE  CYCLE  COSTS 
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1900 
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91 

25050 
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355 
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2020 
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0 
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0 
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8 
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0 
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0 
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8 

0 

0 
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0 
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2008 
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0 
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195 

15547 
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15 

0 

1012 
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0 

15602 

38283 

2009 

0 

0 

2647 

207 
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556 

2502 
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FIGURE  4 


The  proposed  development  procurement  was  for  engineering 
design  services  to  modify  the  existing  ADAPS  data  package  for 
SETS  requirement,  a  deliverable  level  3  data  package  of  the 
modified  system,  and  a  prototype  system  to  verify  the  design. 

The  level  3  data  package  acquired  will  ultimately  be  used  for 
procurement  of  follow  on  production.  The  follow  on  production 
units  are  expected  to  be  procured  through  open  competition.  The 
milestone  chart  of  the  SETS  is  provided  in  Appendix  D. 

Life  cycle  cost  refers  to  all  costs  associated  with  a  system 
throughout  a  defined  life  cycle.  The  life  cycle  for  the  study  of 
the  SETS  began  in  1990  with  the  developmental  phase.  Production 
is  scheduled  to  start  in  1994.  Operation  and  support  will  be  15 
years  beginning  1994.  Life  cycle  cost  definitions  are  given  as 
Appendix  C. 

6.  STATUS  QUO  COST 

The  methodology  for  determining  the  status  quo  LCC  is 
provided  as  appendix  A  of  this  report.  A  combination  of  analogy 
and  parametric  based  techniques  were  used.  Results  of  the 
estimate  are  given  as  figure  5  below. 
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820 
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254 
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Figures  6  and  7 

give  the  quantities  of  existing  test 

systems 

in  use  today.  These 

figures  do  not 

include 

base  closures 

and 

carrier  decommissionings . 
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CARRIER  BASED  ETS  INVENTORY 


FIGURE  7 


Fuel  Cost  Savings 

To  determine  the  potential  fuel  cost  savings  of  the  SETS 
over  the  existing  ETS,  the  following  informations  and  assumptions 
as  documented  in  NAEC-MISC-52-1055  were  utilized: 

1)  The  SETS  run  time  will  be  3.1  hr/run.  (includes  set  up  & 
break  down  time) 

2)  The  existing  test  system  run  time  is  4.3  hr/run.  (includes 
set  up  &  break  down  time) 

3)  The  Total  Items  Processed  (TIP)  is  66,960. 

4)  There  are  279  existing  test  cells. 

5)  132  SETS  are  to  be  manufactured. 

6)  The  SETS  will  replace  existing  units  in  a  2.5:1  ratio. 

7)  The  average  aircraft  engine,  fuel  flow  turbine  will  consume 
2.8  G PM/168  GPH/1142 . 4  PPH. 

8)  Inflation  rates  generated  by  the  cost  model  were  applied. 

Data  for  existing  land  based  test  systems  was  used  to  calculate 
average  values.  According  to  NAEC-DESDAT-52-742,  the  mean  test 
time  of  the  SETS  will  be  1.1  hours.  The  existing  test  systems 
have  a  mean  test  time  of  2.2  hours.  This  represents  a  50% 
reduction  in  test  time  for  the  SETS.  Using  the  aforementioned 
informations  and  assumptions,  the  yearly  fuel  usage  rates  were 
calculated.  It  is  estimated  that  by  utilizing  the  SETS  in  the  20 
year  life  cycle  would  result  in  a  fuel  cost  savings  of 
approximately  $501,530,000.00. 
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7.  SETS  DECISION 

As  can  be  seen  in  figure  8  below,  there  is  a  significant 
cost  savings  to  the  Navy  by  retiring  the  existing  test  systems 
and  procuring  the  SETS.  The  need  for  the  Standard  Engine  Test 
System  (SETS)  is  twofold.  First,  the  Navy  calls  for  fleet  I- 
level  maintenance  activities  to  have  out-of-airframe  engine  runup 
capabilities;  various  engine  test  systems  are  currently  in  the 
inventory  to  fulfill  this  requirement.  In  accordance  with  the 
engine  test  system  Program  Element  Master  Plan's  (PEMP) ,  the 
majority  of  existing  support  equipment  used  for  out-of-airframe 
engine  testing  will  have  exceeded  its  useful  service  life  by  the 
year  2000.  Subsequently,  it  is  more  cost  effective  to  provide 
Fleet  activities  with  new  engine  test  equipment  rather  than 
support  and  upgrade  out  dated  engine  test  equipment  currently  in 
the  inventory.  Secondly,  the  technology  of  engines  now  being 
bought  for  the  Fleet  (i.e.  F110-GE-400.  F404-GE-F5D2 ,  T406-AD- 
400,  etc.)  demands  digital  signal  interface  between  the  engine 
test  system  and  the  engine  control  unit  (ECU) .  All  existing 
engine  test  equipment  process  only  analog  signals.  Upgrades  to 
provide  digital  signal  processing  (see  section  on  adaptor 
assemblies  in  this  report)  is  not  cost  effective.  The  total  LCC 
savings  realized  by  procuring  the  SETS  is  estimated  to  be 
$861,807,000. 
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STATUS  QUO  VS.  SETS 
LIFE  CYCLE  COST  BREAKDOWN 
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FIGURE  8 

8.  SETS  TEST  PLAN 

The  SETS  has  been  demonstrated  using  a  APU  as  the  test 
engine.  Future  testing  will  be  accomplished  using  a  F-404 
turbofan  and  T-700  turboshaft  engines.  In  addition,  a 
maintainability  demonstration  will  be  done.  After  the  three 
engines  are  tested,  the  SETS  will  undergo  TECHEVAL  at  Patuxent 
River.  Results  from  the  testing  will  be  used  to  further  validate 
the  original  LCC  analyses  done. 

9.  INITIAL  FEEDBACK  OF  SETS  LCC 

Initial  testing  with  the  APU  on  the  SETS  indicated  that  the 
factors  utilized  to  generate  the  LCC  on  the  SETS  were  consistent 
with  the  results  achieved.  The  SETS  will  provide  a  substantial 
savings  on  fuel.  Because  of  the  automated  data  acquisition 
system  on  the  SETS,  engine  runs  will  not  be  as  long.  Warm-up 
time  for  the  engine  will  also  be  shorter.  At  the  present  time, 
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the  technical  publication  for  the  engine  requires  a  five  minute 
warm-up  time  for  the  engine.  The  instrumentation  on  the  SETS 
shows  that  the  engine  actually  stabilizes  and  is  ready  for 
testing  after  two  minutes.  A  change  to  the  technical 
publications  will  result  in  at  least  a  three  minute  shorter  run 
time  for  each  test.  It  has  also  been  validated  that  a  two  vice 
four  person  crew  will  be  necessary  to  conduct  an  engine  run. 

This  will  save  considerable  manpower.  The  data  acquisition 
system  will  take  all  required  data  points  for  each  engine 
automatically.  This  will  save  additional  fuel. 

Because  the  prototype  SETS  has  been  delivered  and  is  on  site 
at  NAWCADLKE  it  will  be  easy  to  get  additional  LCC  data.  As 
NAWCADLKE  is  CFA  for  the  SETS,  additional  information  is  readily 
available  from  the  project  engineer,  logistician,  TECHEVAL, 
maintenance  demonstration,  and  First  Article  Test  (FAT)  report. 
This  is  the  only  reasonable  and  cost  effective  way  to  validate 
the  SETS  LCC.  The  following  charts  show  the  time  frames  and 
magnitudes  of  the  main  cost  categories  which  require  validation. 
Obviously,  this  will  be  a  continuing  effort.  Figures  9  through 
12  depict  the  results  of  the  original  LCC  estimate. 
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10.  ANALYSIS  &  COMPARISON  OF  EXPECTED/ ACTUAL  LCC 

Due  to  their  relative  importance,  there  are  three  main  cost 
drivers  in  the  LCC  for  the  SETS  which  should  be  validated.  They 
should  be  validated  because  they  are  the  main  cost  drivers.  In 
addition,  they  are  easy  to  validate.  Fuel  costs  for  the  APU  were 
within  an  acceptable  range  of  the  initial  estimate.  As 
projected,  manpower  cost  savings  were  realized  by  using  a  two 
vice  four  man  crew  for  engine  testing.  This  was  validated  during 
the  APU  testing.  The  estimated  acquisition  costs  will  be 
validated  after  final  contract  award.  A  recent  contract  award 
for  SETS  pre-production  showed  that  the  cost  paid  for  each  SETS 
was  within  10%  of  the  projected  development  cost. 

11.  LESSONS  LEARNED 

The  LCC  for  the  SETS  was  done  in  a  cost  effective  manner 
because  the  cost  analyst  was  on  site  where  the  engineering  work 
was  performed.  The  cost  analyst  was  a  specialist  in  Support 
Equipment  (SE)  and  knew  the  technical  personnel  involved  in  the 
project.  This  gave  the  LCC  analysis  a  shorter  lead  time  with 
fewer  errors.  It  also  permitted  the  cost  analyst  to  develop  Cost 
Estimating  Relationships  (CER)  for  use  on  future  SE  LCC  analyses. 

It  is  important  to  keep  the  main  focus  on  the  follow  up  for 
the  LCC  analysis  on  the  major  cost  drivers.  In  the  case  of  the 
SETS,  the  authors  selected  the  top  three  cost  drivers.  This  will 
continue  to  be  important  in  a  period  of  a  lack  of  funding  and  a 
shortage  of  qualified  cost  analysts. 

Information  gained  from  a  similar  Air  Force  program  was 
invaluable  in  accurately  determining  LCC  for  the  SETS.  At  the 
time  that  the  LCC  was  performed,  there  was  no  existing  Navy 
equipment  to  base  the  analysis  on. 

The  initial  testing  reports  were  important  in  determining 
that  the  SETS  LCC  was  on  track.  When  the  SETS  is  fielded, 
additional  information  can  be  derived  from  maintenance  data  for 
the  test  system  and  the  actual  engines.  The  FAT  report  provided 
additional  valuable  information  to  validate  the  SETS  LCC 
analysis. 


15 


12.  CONCLUSION 

The  LCC  for  the  SETS  was  a  main  driver  in  the  decision  to  go 
ahead  with  a  new  engine  testing  program.  The  LCC  is  a  important 
tool  to  make  intelligent  management  decisions.  It  is  important 
to  continue  to  validate  each  LCC  as  it  is  completed  to  ensure 
that  the  customer  continues  to  receive  good  information. 

The  SETS  program  is  also  one  of  the  first  demonstrations  of 
a  successful  LCC  and  SED  process.  It  is  important  to  remember 
that  the  LCC  is  an  important  input  in  the  COEA. 

There  were  no  major  discrepancies  found  in  the  SETS  LCC. 

The  authors  estimate  that  90%  of  work  that  should  go  into  the  LCC 
is  the  initial  data  gathering.  It  was  easy  to  validate  the  LCC 
because  of  the  relatively  short  lead  time  for  SE.  All  the 
engineers  and  logisticians  remain  available  to  validate  the  LCC 
results  because  of  the  short  lead  time. 

There  is  a  need  to  continually  validate  the  results  for  the 
LCC.  Program  managers  should  be  encouraged  to  fund  validations 
of  each  LCC  performed.  This  will  result  in  a  continuous 
improvement  of  the  LCC  reports  submitted.  It  will  also  foster 
pride  in  a  job  well  done  for  the  individual  analyst. 
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APPENDIX 


A 


METHODOLOGY  OF  E8TIMATE 

The  Parametric  Review  of  Information  for  Costing  and 
Evaluation  (PRICE)  cost  model  was  used  to  estimate  the 
development  and  production  costs  for  the  SETS,  as  well  as  the 
life  cycle  cost  estimates  for  the  SETS  and  the  current  ETS. 
Parametric  is  a  computerized  cost  estimating  system  designed  to 
calculate  engineering  and  production  factors.  Application  of  the 
model  requires  obtaining  input  data  based  upon  physical  aspects 
of  the  item  such  as  size  and  weight,  qualitative  aspects  related 
to  manufacturing  complexities,  other  inputs  based  on  procurement 
quantities,  schedules,  economics,  engineering  efforts  and  systems 
integration.  Parametric  cost  estimating  is  used,  both  by 
contractors  and  the  government,  for  several  different  purposes  in 
acquisition  planning  and  execution  (ref.  DCAA  Contract  Audit 
Manual  9-1002.3  1987).  Parametric  models  have  been  used  by  U.S. 
Government  agencies  to  estimate  development/production  cost,  life 
cycle  cost,  and  schedules  since  1975.  Expert  opinion,  mission 
requirements  and  historical  information  on  similar  test  equipment 
(i.e.  P107  Cruise  Missile  Program  ATE  RFQ  Williams  International 
GLB-ASE-6125)  were  also  employed  to  compare  technical  and  cost 
characteristics.  In  addition  the  Cost  Analysis  Strategy 
Assessment  (CASA)  model  was  also  utilized  as  a  supplement  to 
estimate  the  life  cycle  cost  of  the  SETS.  The  CASA  is  an  analogy 
based  cost  estimating  model  which  is  a  derivative  of  Honeywell's 
Total  Resource  and  Cost  Evaluation  (TRACE)  family  of  logistics 
and  life  cycle  cost  models. 
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APPENDIX  B 


ACRONYMS 


ACRONYM 


DEFINITION 


ACAT 

ADAPS 

APN 

APU 

CA/E 

CASA 

CDRL 

CER 

CFA 

CMP 

COEA 

DCAA 

DoD 

ECM 

ECU 

FAT 

ETS 

FCM 

FY 

GPH 

GPM 

GTC 

I 

I&C 

ICATS 

ILS 

ILSDS 

LCC 

LSA 

MTBF 

MTTR 

NAWCADLKE 

OJT 

PC 

PEMP 

PR 

PRICE 

RAMS 

SECM 

SED 

SETS 

SML 

TIP 

TURN 

TRACE 


Acquisition  Category 

Automatic  Data  Acquisition  and  Processing  System 

Aircraft  Procurement  Navy 

Auxiliary  Power  Unit 

Cost  Analysis/Estimating 

Cost  Analysis  Strategy  Assessment 

Contract  Data  Requirements  List 

Cost  Estimating  Relationship 

Cognizant  Field  Activity 

Cruise  Missile  Program 

Cost  and  Operational  Effectiveness  Analysis 

Defense  Contract  Audit  Agency 

Department  of  Defense 

Engine  Control  Module 

Engine  Control  Unit 

First  Article  Test 

Engine  Test  System 

Facilities  Control  Module 

Fiscal  Year 

Gallons  Per  Hour 

Gallons  Per  Minute 

Gas  Turbine  Compressor 

Intermediate 

Instrumentation  and  Control 

Integrated  Computer  Automated  Test  System 

Integrated  Logistic  Support 

ILS  Detail  Specification 

Life  Cycle  Cost 

Logistic  Support  Analysis 

Mean  Time  Between  Failures 

Mean  Time  To  Repair 

Naval  Air  Warfare  Center  Aircraft  Division  Lakehurst 

On  Job  Training 

Personal  Computer 

Program  Element  Master  Plan 

Purchase  Request 

Parametric  Review  of  Information  for  Costing  & 
Evaluation 

Reliability,  Availability,  Maintainability,  Safety 

Soft  Engine  Control  Module 

Support  Equipment  Decision 

Standard  Engine  Test  System 

Support  Material  List 

Total  Items  Processed 

Technical  Logistics  Reference  Network 

Total  Resource  and  Cost  Evaluation 
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APPENDIX 


C 


LIFE  CYCLE  DEFINITIONS 

Development  cost  -  engineering  design;  design  documentation; 
reliability  and  maintainability;  quality  assurance;  data; 
software;  test  and  evaluation  of  prototype;  and  associated 
management  functions. 

Production  cost  -  manufacturing  cost;  production  operations; 
quality  control;  logistics  support  requirements;  and  associated 
management  functions. 

Operational  and  support  cost  -  operation  of  the  system  in  the 
field;  logistic  support;  supply;  transportation  and  handling; 
technical  data;  facilities;  training;  modifications;  etc. 

Disposal  and  retirement  -  disposal  of  non-repairable  items; 
retirement  of  the  system. 

Equipment/Development  -  The  equipment  development  cost  is  a 
summation  of  research  and  development  costs,  including  design, 
drafting,  systems  engineering,  project  management,  data 
submittals,  prototype  development  and  fabrication,  engineering 
shop  tooling,  and  test  equipment.  The  engineering  data  primarily 
consists  of  the  standard  maintenance  and  operating  manuals,  and 
the  usual  Contract  Data  Requirements  List  (CDRL)  data 
requirements. 

Equipment/production  -  The  equipment  production  cost  includes  the 
total  acquisition  cost  of  all  equipment  deployed  over  the  life  of 
the  program  and  the  non-recurring  production  costs.  The 
equipment  unit  cost  is  a  function  of  the  average  recurring 
production  cost,  the  reference  quantity,  the  learning-curve 
effect,  and  the  total  quantity  of  equipment  deployed  plus  initial 
equipment  spares.  Production  for  the  SETS  is  scheduled  to  start 
in  1994.  132  SETS  are  proposed  to  be  built. 

Supply  Spares  and  Repair  Parts  -  Spare  parts  are  defined  as 
product  for  separate  production  and  procurement  that  are  required 
for  maintenance  or  repair.  This  would  exclude  end  items  which 
have  redundancy  designed  into  the  product  to  avoid  failure. 

Spare  parts  could  range  from  piece  parts  to  minor  assemblies  to 
end  items.  They  are  procured  from  the  manufacturer  to  replace 
like  equipment  that  is  lost  from  the  program  as  a  result  of  scrap 
or  attrition.  Initial  spares  are  procured  concurrently  with  the 
production  of  the  basic  equipment.  All  parts  are  considered  non- 
repairable  and  therefore  they  are  handled  as  scrap.  Production 
scrap  and  waste  are  not  included  in  spare  parts. 
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Supply  Administration/Production  and  support  -  The  supply 
administration  cost  for  production  is  the  cost  to  enter  new  spare 
items  into  the  supply  system.  The  items  may  include  non-standard 
parts,  module  types,  and  equipments.  The  supply  administration 
cost  for  support  is  the  cost  to  retain  new  items  mentioned  above 
in  the  supply  system  over  the  life  of  the  program.  Using  the 
value  acquired  from  the  Naval  Air  Systems  Command  Default  Data 
Guide,  the  cost  to  enter  new  spare  items  is  $701.00  and  the  cost 
to  retain  them  is  $199.00. 

Manpower  and  Corrective  Maintenance  -  The  manpower  cost  for 
support  is  the  cost  of  labor  required  for  set-up,  take-down, 
calibration,  operating  and  maintaining  the  equipment  over  the 
life  of  the  program.  It  is  assumed  that  each  technician  is 
assigned  exclusively  to  one  system.  The  labor  rates  are  assigned 
to  the  location  at  which  the  maintenance  is  performed  without 
regards  to  the  rank  or  skill  of  the  technician.  For  a  team  of 
operators  and  technicians,  the  labor  rates  should  be  multiplied 
by  the  number  of  people  per  team. Preventive  Maintenance 

Supply  Facility  -  The  supply  storage  costs  are  based  on  the 
monthly  storage  rates  and  the  supply  hardware  volumes.  It  would 
cost  $0. 60/cuft/mo  to  store  spare  modules  or  parts  in  the  land- 
based  and  carrier-based  maintenance  levels. 

Shipping  -  The  shipping  costs  are  computed  from  the  weight  of 
each  stock  item  shipped  and  number  of  trips.  Repairable 
equipments  and  modules  are  charged  for  a  return  trip  while 
balance  consumed  spares  are  charged  only  for  one  way  trips  from 
the  manufacturer  down  towards  the  equipment  level.  The  only 
shipping  cost  to  be  considered  for  the  SETS  is  the  cost  to  ship 
the  units  from  the  Contractor  to  the  Equipment  Location.  The 
costs  to  ship  the  units  from  one  maintenance  location  to  another 
is  not  relevant. 

Contractor  Support  and  Training  -  The  contractor  support  cost  is 
generated  by  using  the  following  cost  factors:  (1)  contractor 
cost  for  equipment?  and  (2)  contractor  cost  for  module  repair. 

The  contractor  support  cost  for  technical  evaluation  will 
primarily  involve  the  cost  to  correct  any  flaws  found  in  the 
system.  The  remainder  of  the  training  process  will  be  an  On  Job 
Training  (OJT) .  The  OJT  cost  is  included  in  the  manpower  cost. 
However,  due  to  normal  attrition  and  the  fact  that  an  operator 
may  decide  not  to  re-enlist  in  the  service,  consideration  must  be 
taken  as  to  the  number  of  new  operators  to  be  trained  for  this 
particular  system.  There  is  an  operator  turnover  every  three 
years.  Therefore,  it  can  be  concluded  that  the  yearly  turnover 
rate  is  33%.  The  plans  and  report  on  personnel  and  training 
manpower  are  included  in  the  development  phase.  The  cost 
incurred  in  this  category  is  part  of  the  Contract  Date 
Requirements  List  (CDRL) . 

Modifications/ECP's  -  It  is  anticipated  that  there  could  be  three 
modifications  for  the  SETS  to  make  allowance  for  new  technology 
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and  software  upgrades.  The  modif ications/ECP's  are  expected  to 
take  place  in  the  6th,  10th,  and  14th  year  after  production  ends. 

Additional  software  Program  -  The  software  program  cost  includes 
the  development  of  software  for  additional  engines.  The  CA/E 
Office  estimated  2  man-years  per  year  as  a  basis  for  the  cost 
from  year  1995  to  1999,  and  1  man-year  per  year  from  year  2000  to 
2009. 

Integrated  Logistics  Support  Monitor  -  An  estimated  cost  of 
$400,000.00  APN-7  funding  would  be  required  to  properly  review 
and  verify  all  the  ILS  items  throughout  the  development  phase  and 
the  first  year  of  production  prior  to  their  acceptance  by  the 
Navy. 
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APPENDIX  D 


8ET8  ASSUMPTIONS 


AREAS  OF  CONSIDERATION 

The  initial  shipping  cost  for  the  SETS  prototype  is  $35K; 
this  value  was  obtained  from  the  ASE  estimate.  The  majority  of 
the  shipping  cost  is  shown  concurrently  with  the  production  of 
the  SETS.  The  cost  involves  shipment  of  the  SETS  and  cables  from 
contractor  to  their  designated  destinations.  The  remainder  of 
the  shipping  cost  is  the  cost  to  ship  spares  modules  and  parts. 

It  is  assumed  that  all  maintenance  actions  will  be  done  by 
military  personnel.  An  average  maintenance  action  takes  1.5 
hours  with  an  estimated  preventive  maintenance  of  four  (4)  per 
month. 

On-Site  Calibration  -  A  calibration  team  will  be  contracted  by 
the  Navy  to  perform  an  on-site  calibration  test  on  each  test  cell 
every  6  months  which  produces  2  calibrations  in  one  (1)  year  for 
one  (1)  cell.  Each  test  would  cost  $3000  as  per  the  Technical 
Evaluation  Facility  from  Patuxent  River. 

The  SETS  will  be  used  at  I  level  aviation  maintenance  activities 
worldwide,  both  ashore  and  afloat,  to  automatically  control  out- 
of-airframe  engine  test  runs.  There  will  be  two  operators 
assigned  per  test  cell.  Class  E-5  and  above  operators  will  be 
paid  $12. 36/hr  to  operate  the  SETS.  This  labor  rate  is  a 
weighted  average  rate  obtained  from  the  Naval  Air  Systems  Command 
Default  Data  Guide.  Regular  military  personnel  will  be 
maintaining  the  systems;  a  repair  cost  of  $12. 36/hr  is 
applicable  to  the  intermediate  levels.  The  labor  rate  for 
civilian  is  $56. 78/hr  for  depot  rework.  Maintenance  cost 
includes  preventive  and  corrective  maintenance. 

Reliability  and  Maintainability  -  Based  on  an  estimate  by  ASE, 
the  Mean  Time  Between  Failure  (MTBF)  of  the  SETS  should  equal  or 
exceed  1680  hours,  with  a  Mean  Time  To  Repair  (MTTR)  of 
approximately  one  hour  assuming  that  the  Navy  spares  the  major 
assemblies  and  modules.  Clifford  Ash,  of  ASE  has  guaranteed  the 
MTBF  of  1680  hours  given  to  the  CA/E  Office  for  this  cost 
estimate  will  meet  or  exceed  the  acceptance  requirements. 

Usage  Rate  -  An  usage  rate  of  131  hours  per  month  for  the  SETS 
based  on  the  program  manager's  estimate  was  utilized  for  the  life 
cycle  cost  calculation. 

Disposal  Cost  -  It  is  assumed  that  all  SETS  will  still  be  in 
service  at  the  end  of  the  life  cycle  cost  analysis  period. 
Therefore,  the  disposal  cost  is  not  a  factor  in  the  estimate. 
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inflation  Rata  -  An  inflation  rate  factor  of  6.2%  was  applied  in 
the  estimate  for  the  years  of  operation  and  support  period.  For 
it,,  calculation  of  the  factor  the  Consumer  Price  Index,  the 
Producers  Price  Index,  the  International  Monetary  Fund,  McCraw- 
Hill  Data  Resources  Inc.,  and  the  Federal  Reserve  discount  rate 
were  used  as  sources. 
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APPENDIX 


E 


ET8  ASSUMPTIONS 

Analogy  Cost  Rational  -  The  following  Engine  Test  Systems  (ETS) 
have  been  used  as  a  basis  for  the  life  cycle  cost  of  the  existing 
ETS.  Through  the  Technical  Logistics  Reference  Network  (TLRN) 
and  the  logistics  group,  it  has  been  concluded  that  these  ETS 
information  are  accurate  and  may  be  used  as  a  guide  for  the  life 
cycle  cost  analysis  of  the  existing  ETS. 

As  part  of  the  life  cycle  cost  and  value  analysis,  the  CA/E 
office  utilized  four  specific  existing  test  systems  to  represent 
each  of  the  four  major  engine  test  disciplines.  Each  specific 
system  was  carefully  selected  as  representing  the  most  current 
state  of  the  art  technology  utilized  by  the  Navy  today.  In 
addition,  current  purchase  prices  as  well  as  maintenance  and 
support  parameters  were  also  utilized  in  the  development  of  the 
cost  to  operate  the  existing  systems.  The  following  test  systems 
were  chosen  as  benchmarks  representing  the  major  categories: 

APU  TESTING:  The  A/E37T-26A  Auxiliary  Power  Unit  (APU)  test 
stand  was  utilized  as  a  baseline  data  reference  for  all  APU  test 
systems.  As  one  of  the  newest  and  most  up-to-date  APU  testers 
the  T-26A  has  the  most  current  technology  of  any  APU  tester 
utilized  today  in  the  Navy.  All  external  components/modules  such 
as  cables,  hoses  and  fuel  skids  associated  with  the  T-26A  would 
be  considered  as  usable  with  the  SETS.  Current  replacement  costs 
for  the  A/E37T-26A  as  listed  in  the  Technical  Logistics  Reference 
Network  (TLRN)  in  1990  dollars  is  $168,000.00,  various  adaptor 
assemblies  are  also  utilized  in  the  T-26A  and  appear  to  average 
$50,000.00  each.  An  allowance  for  the  adaptor  assemblies  of 
$100,000.00  per  system  would  bring  the  replacement  cost  of  a 
A/E37T-26A  test  system  to  $268,000.00  each  in  1990  dollars. 

TURBOSHAFT  TESTING:  The  A/E37T-24  Turboshaft  test  system  was 
utilized  as  a  baseline  data  reference  for  all  turboshaft  engine 
test  systems.  The  test  system  is  available  in  four  basic 
configurations.  All  configurations  include  an  instrumentation 
and  control  (I&C)  cab  assembly,  a  fuel  and  oil  system 
installation,  an  engine  test  trailer,  a  junction  box,  and  various 
hoses  and  cables  for  interconnecting  system  components.  With 
peculiar  adapters,  which  are  not  supplied  as  part  of  the  basic 
system,  the  T-24  can  be  used  to  test  the  T-64,  T-400,  T-58,  and 
the  T700  engines.  The  T-24  was  used  as  a  representative 
turboshaft  test  system  for  the  analogy  estimate  because  of  the 
types  of  engines  it  supports,  and  the  ability  to  derive  realistic 
MTBF,  MTTR,  and  other  data  on  the  system.  All  external 
components  associated  with  the  T-24  would  be  considered  as 
usable  with  the  SETS.  Current  replacement  costs  for  A/E37T-24 
test  system  as  listed  in  the  TLRN  in  1990  dollars  is  $499,000.00 
dollars  each  ($214,000  for  the  I&C  cab,  $40,800  for  the  fuel 
system,  and  $245,000  for  the  test  trailer). 
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TURBOJET/FAN  TESTING:  The  A/M37T-23  Mobile  Engine  Test  System 
was  used  as  the  baseline  data  reference  for  all  turbo jet/turbof an 
test  systems.  The  T-23  provides  complete  functional  and 
operational  testing  of  turbojet/fan  engines  in  the  Navy  inventory 
using  the  latest  in  precision  electronic  instrumentation.  The 
instrumentation  package  components  in  the  T-23  are  identical  to 
those  in  the  T-10  and  T-14  test  systems,  although  packaged 
differently  for  optimum  mobility.  Therefore,  the  instrumentation 
packages  and  adapter  assemblies  are  compatible  in  general.  All 
external  components  associated  with  the  T-23  would  be  considered 
as  usable  with  the  SETS.  Current  replacement  costs  for  the 
A/M37T-23  as  listed  in  the  TLRN  in  1990  dollars  is  $445,000.00. 
This  cost  is  for  the  complete  system  including  the  I&C  cab,  fuel 
system,  runup  trailers,  and  all  additional  equipment  required  to 
operate  the  test  system. 

TURBOPROP  TESTING:  The  A/E37T-17  turboprop  engine  test  system 
was  used  as  the  baseline  data  reference  for  all  turboprop  test 
systems.  The  basic  test  system  consists  of  an  instrumentation 
and  control  (I&C)  cab  assembly,  a  fuel  skid  and  a  new  or  modified 
Allison  engine  test  bed  and  engine  adaptor  assembly.  All 
external  equipment  and  components  associated  with  the  T-17  would 
be  considered  as  usable  with  the  SETS.  Current  replacement  costs 
for  the  A/E37T-17  test  system  as  listed  in  the  TLRN  in  1990 
dollars  is  $228,000.00  per  unit.  The  actual  cost  breakdown  for 
the  system  is  $102,000.00  for  the  I&C  cab,  $38,000.00  for  the 
fuel  system,  and  $90,000.00  for  the  engine  test  bed. 

OPERATIONAL  COSTS  -  In  order  to  develop  operational  costs,  a 
weighted  average  MTBF  of  680  hours  and  a  MTTR  of  2  hours  were 
utilized  for  the  life  cycle  cost  analysis  of  the  existing  ETS. 

All  corrective  and  preventive  maintenance  actions  have  been 
considered  as  well  as  semi-annual  calibrations.  Usage  rates  have 
been  derived  from  3M,  various  field  activity  logs,  and  interviews 
which  were  then  utilized  to  estimate  the  life  cycle  costs. 
Disposal  cost  will  not  be  a  factor  as  old  test  cells  would  be 
utilized  as  spare  supply  or  as  sale  to  foreign  country. 

Supply  Spares  and  Repair  Parts  -  The  Navy  is  responsible  for  the 
storage  of  the  spare  parts.  At  the  same  time,  they  will  procure 
spares  in  accordance  to  the  needs  of  the  Navy.  This  would  lessen 
the  costs  in  both  storage  and  procurement.  Spares  will  be 
procured  throughout  the  life  of  the  test  set.  It  is  estimated 
that  the  yearly  spares  cost  is  approximately  five  percent  (5%)  of 
the  end  item  cost. 

Supply  Administration/Production  and  Support 

The  cost  for  supply  administration  starts  as  soon  as  the 
supply  spares  and  repair  parts  are  procured,  and  it  goes  to  the 
end  of  the  study  which  is  the  year  2009.  The  cost  to  enter  these 
items  in  the  supply  system  is  $701.00  each  item.  The  cost  to 
retain  each  new  item  in  the  supply  system  is  $199.00.  These 
values  are  acquired  from  the  Naval  Air  Systems  Command  Default 
Data  Guide. 
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Manpower  and  corrective  Maintenance 

The  existing  units  are  used  at  intermediate-level  aviation 
maintenance  activities  worldwide,  both  ashore  and  afloat,  to 
automatically  control  out-of-airframe  engine  test  runs.  It  is 
assumed  that  there  are  four  (4)  operators  assigned  to  each  test 
cell.  The  operators  are  classified  as  Class  E-5  and  above.  The 
weighted  average  labor  rate  for  both  the  carrier-based  and  land- 
based  operators  is  $12.36.  The  labor  rate  was  obtained  from  the 
Naval  Air  Systems  Command  Default  Data  Guide.  On-the-job  (OJT) 
training  is  included  in  the  manpower  cost. 

The  historical  engine  test  time  for  an  ETS  is  4.3  hours  per 
engine  based  on  a  weighted  average  determined  by  an  extensive 
data  search.  20  tests/mo/cell  for  279  cells  equals  to  66,960 
engines  tested.  The  number  of  test  performed  by  carrier  based 
ETS  have  not  been  listed  completely;  therefore,  the  approximate 
usage  rate  has  been  calculated  using  the  information  presented 
for  the  land-based  ETS. 

Regular  military  personnel  will  be  maintaining  the  systems; 
therefore  a  repair  cost  of  $12. 36/hour  is  still  applicable  to  the 
intermediate  level.  This  rate  is  also  obtained  from  the  Naval 
Air  Systems  Command  Default  Data  Guide.  In  case  a  module  cannot 
be  repaired  in  the  maintenance  level,  the  module  will  be  sent  to 
the  depot  level  which  is  a  highly  specialized  repair  facility. 

It  is  assumed  that  maintenance  personnel  and  spares  are  on  hand. 

Preventive  Maintenance 

It  is  assumed  that  all  maintenance  actions  will  be  done  by 
military  personnel.  An  average  maintenance  action  takes  1.5 
hours  with  an  estimated  preventive  maintenance  of  4  per  month. 

On-Site  Calibration 

A  calibration  team  will  be  contracted  by  the  Navy  to  perform 
an  on-site  calibration  test  on  each  test  cell  every  6  months 
which  produces  2  calibrations  in  one  (1)  year  for  one  (1)  cell. 
Each  test  would  cost  $3000  as  per  the  Technical  Evaluation 
Facility  from  Patuxent  River. 

Supply  Facility 

The  cost  to  store  spare  modules  and  parts  on  land  or  on 
board  is  approximately  $0.60/cuft/mo.  This  figure  was  acquired 
from  the  Naval  Air  Systems  Command  Level  of  Analysis  Default  Data 
Guide. 
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